Repository navigation
docs: add a figure of compute saving against cache size - #2
Merged
Merged
Conversation
The README's table gives one cache size. The figure gives five, from 16k to 256k blocks (24 to 384 GiB), for the same four policies and cost model on both production traces: compute-aware eviction gains most when the cache is tight, and on freeinference every policy converges by 64k blocks. doc/figures/ holds the figure, in a light and a dark variant, the prefixsim RESULT lines it is plotted from, and the script that plots them, which also says how to rerun the 40 simulations. The 16k freeinference results match the README's table. The chutes 256k RandomCompute run is still going; its point follows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VD7L7yjfBMktJRmHXQGHD
#1 merged with the logo only; the figure commit was pushed after the merge. Bring the branch up to main so the figure can go in on its own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VD7L7yjfBMktJRmHXQGHD
The chutes 256k RandomCompute run finished: compute saving 0.7148, against LRU's 0.7150 and S3-FIFO's 0.7093, so the README's reading of chutes holds at every cache size. All 40 runs report no unexpected evictions and no skipped requests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015VD7L7yjfBMktJRmHXQGHD
LauYeeYu
approved these changes
Nov 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the results figure to the README. It was meant to be part of #1, but that PR merged with the logo only, before the figure was pushed.
Figure
The figure plots compute saving ratio against cache size, below the README's policy table. It covers 16k to 256k blocks (24 to 384 GiB) on both production traces, for LRU, S3-FIFO, RandomCompute and the BeladyCompute oracle, with the
qwen3coder30b_blksz_16cost model. GitHub's dark theme gets its own version through a<picture>element.A short paragraph under the figure states what it shows:
Files
doc/figures/contains:compute_saving.svgandcompute_saving-dark.svg: the figure, in a version for each GitHub theme.compute_saving.txt: theprefixsimRESULTlines of all 40 runs, which the figure is plotted from.plot_compute_saving.py: the plotting script. It needs matplotlib, and its docstring has the loop that reruns all 40 simulations.How the data was checked
prefixsimrun on the Hugging Face traces.n_unexpected_eviction=0andn_req_skipped=0.🤖 Generated with Claude Code
https://claude.ai/code/session_015VD7L7yjfBMktJRmHXQGHD